Field of the Invention
The present invention relates to a processing apparatus, a processing system, an image pickup apparatus, a processing method, and a non-transitory computer-readable storage medium.
Description of the Related Art
Obtaining more physical information regarding an object can generate images based on a physical model in image processing after imaging. For example, an image where visibility of the object is changed can be generated. Visibility of the object is determined on the basis of information such as shape information of the object, reflectance information of the object and light source information. As physical behavior of reflected light that is emitted from alight source and is reflected by the object depends on a local surface normal, using not a three-dimensional shape but the surface normal of the object as shape information is especially effective. As a method obtaining the surface normal of the object, for example, a method that converts a three-dimensional shape calculated from distance information obtained using a method such as triangulation using laser light and a twin-lens stereo into surface normal information is known. However, such a method complicates the structure of the apparatus, and accuracy of the obtained surface normal is insufficient.
In Japanese Patent Laid-Open No. 2010-122158 and “Photometric stereo” (A research report of Information Processing Society of Japan, Vol. 2011-CVIM-177, No. 29, pp. 1-12, 2011) by Yasuyuki Matsushita, a photometric stereo method is disclosed as a method obtaining the surface normal of the object directly. In the photometric stereo method, reflectance characteristics of the object based on the surface normal of the object and a direction from the object to the light source is assumed, and the surface normal is calculated from luminance information of the object at a plurality of light source positions and the assumed reflectance characteristics. The reflectance characteristics of the object can be, for example, approximated using a Lambert reflection model in dependence upon a Lambert's cosine law.
In an image pickup apparatus such as a digital camera, when surface normal information of the object is obtained using the photometric stereo method, a plurality of pieces of luminance information at a plurality of wavelengths (colors) such as red information (hereinafter referred to as “Rch.”), green information (hereinafter referred to as “Gch.”) and blue information (hereinafter referred to as “Bch.”) are obtained. The obtained luminance information depends on color information of the object and the light source, and thus has a S/N ratio different for each wavelength (each color). Accordingly, influence of noise on the luminance information differs for each wavelength (each color), and as a result, variability of obtained surface normal information differs for each wavelength (each color). For example, when a red object is imaged for each of wavelengths of Rch., Gch. and Bch., in Rch., a S/N ratio is high and variability of surface normal information is small, but, in Gch. and Bch., a S/N ratio is low and variability of surface normal information is large. That is, surface normal information is required to be calculated using a proper wavelength (a proper color) based on the obtained luminance information. However, Japanese Patent Laid-Open No. 2010-122158 and “Photometric stereo” (A research report of Information Processing Society of Japan, Vol. 2011-CVIM-177, No. 29, pp. 1-12, 2011) by Yasuyuki Matsushita fail to disclose an obtaining method of surface normal information using a proper wavelength (color) based on the obtained luminance information.